AutoML Leaderboard

Best model	name	model_type	metric_type	metric_value	train_time
	1_Baseline	Baseline	auc	0.5	0.9
	2_DecisionTree	Decision Tree	auc	0.618115	9.84
	3_Default_Xgboost	Xgboost	auc	0.619062	25.89
	4_Default_NeuralNetwork	Neural Network	auc	0.628768	3.74
	5_Default_RandomForest	Random Forest	auc	0.631465	12.84
the best	Ensemble	Ensemble	auc	0.631927	0.76

AutoML Performance

AutoML Performance Boxplot

Features Importance

Spearman Correlation of Models

Summary of 4_Default_NeuralNetwork

<< Go back

Neural Network

• n_jobs: -1
• dense_1_size: 32
• dense_2_size: 16
• learning_rate: 0.05
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

auc

Training time

2.9 seconds

Metric details

	score	threshold
logloss	0.667389	nan
auc	0.628768	nan
f1	0.662236	0.257732
accuracy	0.596849	0.461771
precision	0.58296	0.461771
recall	1	0.118667
mcc	0.194762	0.461771

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.667389	nan
auc	0.628768	nan
f1	0.600809	0.461771
accuracy	0.596849	0.461771
precision	0.58296	0.461771
recall	0.619785	0.461771
mcc	0.194762	0.461771

Confusion matrix (at threshold=0.461771)

	Predicted as 0	Predicted as 1
Labeled as 0	1006	744
Labeled as 1	638	1040

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 3_Default_Xgboost

<< Go back

Extreme Gradient Boosting (Xgboost)

• n_jobs: -1
• objective: binary:logistic
• eta: 0.075
• max_depth: 6
• min_child_weight: 1
• subsample: 1.0
• colsample_bytree: 1.0
• eval_metric: auc
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

auc

Training time

24.9 seconds

Metric details

	score	threshold
logloss	0.671266	nan
auc	0.619062	nan
f1	0.665819	0.346389
accuracy	0.588098	0.464941
precision	0.62963	0.665024
recall	1	0.0809205
mcc	0.179022	0.464344

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.671266	nan
auc	0.619062	nan
f1	0.602029	0.464941
accuracy	0.588098	0.464941
precision	0.571123	0.464941
recall	0.636472	0.464941
mcc	0.178889	0.464941

Confusion matrix (at threshold=0.464941)

	Predicted as 0	Predicted as 1
Labeled as 0	948	802
Labeled as 1	610	1068

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Decision plots

<< Go back

Summary of 1_Baseline

<< Go back

Baseline Classifier (Baseline)

• n_jobs: -1
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

auc

Training time

0.2 seconds

Metric details

	score	threshold
logloss	0.692927	nan
auc	0.5	nan
f1	0.657266	0.440414
accuracy	0.489498	0.440414
precision	0.489498	0.440414
recall	1	0.440414
mcc	0	0.440414

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.692927	nan
auc	0.5	nan
f1	0.657266	0.440414
accuracy	0.489498	0.440414
precision	0.489498	0.440414
recall	1	0.440414
mcc	0	0.440414

Confusion matrix (at threshold=0.440414)

	Predicted as 0	Predicted as 1
Labeled as 0	0	1750
Labeled as 1	0	1678

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of Ensemble

<< Go back

Ensemble structure

Model	Weight
4_Default_NeuralNetwork	1
5_Default_RandomForest	2

Metric details

	score	threshold
logloss	0.664883	nan
auc	0.631927	nan
f1	0.664653	0.332404
accuracy	0.600642	0.515354
precision	0.661448	0.635542
recall	1	0.0855096
mcc	0.200412	0.515354

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.664883	nan
auc	0.631927	nan
f1	0.564152	0.515354
accuracy	0.600642	0.515354
precision	0.605605	0.515354
recall	0.52801	0.515354
mcc	0.200412	0.515354

Confusion matrix (at threshold=0.515354)

	Predicted as 0	Predicted as 1
Labeled as 0	1173	577
Labeled as 1	792	886

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 2_DecisionTree

<< Go back

Decision Tree

• n_jobs: -1
• criterion: gini
• max_depth: 3
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

auc

Training time

8.9 seconds

Metric details

	score	threshold
logloss	0.670126	nan
auc	0.618115	nan
f1	0.661785	0.240481
accuracy	0.596266	0.543829
precision	0.630435	0.65111
recall	1	0.0412844
mcc	0.191693	0.543829

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.670126	nan
auc	0.618115	nan
f1	0.555841	0.543829
accuracy	0.596266	0.543829
precision	0.602225	0.543829
recall	0.516091	0.543829
mcc	0.191693	0.543829

Confusion matrix (at threshold=0.543829)

	Predicted as 0	Predicted as 1
Labeled as 0	1178	572
Labeled as 1	812	866

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Decision plots

<< Go back

Summary of 5_Default_RandomForest

<< Go back

Random Forest

• n_jobs: -1
• criterion: gini
• max_features: 0.9
• min_samples_split: 30
• max_depth: 4
• eval_metric_name: auc
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

auc

Training time

11.9 seconds

Metric details

	score	threshold
logloss	0.664804	nan
auc	0.631465	nan
f1	0.664857	0.322972
accuracy	0.600058	0.521104
precision	0.661448	0.633334
recall	1	0.0407189
mcc	0.199213	0.521104

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.664804	nan
auc	0.631465	nan
f1	0.56407	0.521104
accuracy	0.600058	0.521104
precision	0.604635	0.521104
recall	0.528605	0.521104
mcc	0.199213	0.521104

Confusion matrix (at threshold=0.521104)

	Predicted as 0	Predicted as 1
Labeled as 0	1170	580
Labeled as 1	791	887

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Decision plots

<< Go back